From time to time I need to edit attributes in the MAPublisher MAP Attributes table. Typically this is for formatting the names of features before generating labels, but not always. Often I will figure something out, do the thing, move on, and then kick myself 6 months or 3 years down the road when I have to figure the same thing out all over again. I’m going to write them down here and maybe I won’t have to look them up in the future! And maybe it will help someone else.
Append a string if another string doesn’t already exist
I was working with the WPDA dataset and labeling a subset of the polygons. In this case I wanted to label all of the National Parks. Some of the values in the NAME field had “National Park” somewhere in the name, sometimes at the beginning, sometimes at the end.
Some values have “National Park”, others don’t
So, we need an expression that looks to see if the string “National” exists. If it does, leave the NAME value alone, if it doesn’t, then append “National Park” to the end of the existing value.
(1) If the value in NAME contains “National”, (2) then replace with NAME (ie-leave it alone., (3) if it doesn’t contain “National”, then replace the value with the existing NAME value and append the string “National Park”.
Apply the expression via the “Apply Expression” option in the flyout menu of the MAP Attributes table:
And the features in question have the correct string, “National Park”, applied to them:
I had a access to a smartboard, so I had all of the students gather around (away from their computers) to go over the concepts from Day 3 in a group format. I asked them answer questions about menus and specific purposes of certain functions.
Student Project Work Time
Continue with personal project work and one-on-one help.
Hopefully all students are ready to print their final map at the end of the day. Since I only had a 8.5×11″ printer in the lab, I opted to have all of the students save a final .tif image at 13×19″ and took it home to print so that they had a larger print to show on Day 5.
Day 5:
Overview
Day 5 was just 3 hours long, with a show-and-tell being the last hour. Hopefully the projects are all done, so I did an overview of QGIS concepts learned during the week and explored some of the datasets that the students used for their personal projects. Ask questions about processing, labeling, styling, attributes, etc.
Presentations
Parents and others came to the lab and each student presented their project map. I asked them questions about how they processed the data, why they styled it the way they did, what problems they had, etc.
I had a access to a smartboard, so I had all of the students gather around (away from their computers) to go over the concepts from Day 2 in a group format. I asked them answer questions about menus and specific purposes of certain functions.
Vector Selection and Analysis – Hydro
Have the students load a far more complicated dataset for analysis. I used the Montana state extract for the USGS NHD (National Hydrography Dataset). Use this opportunity to talk about data scale compared to NaturalEarthData that they have been using.
add NHD Flowline, HUC8
Have the students select and create a memory layer for a specific HUC8 boundary
Clip the statewide flowline dataset to the HUC8 memory layer GDAL>Vector Geoprocessing>Clip vector by mask layer, save as memory layer
Have the students use ‘select by expression’ to isolate a specific river, for example: “GNIS_Name” = ‘Bitterroot River’
Copy the selected features>Paste them as a new memory layer
Calculate the statistics on the new River layer (Vector>Analysis Tools>Calculate Basic Statistics..) and ask the students which statistic tells us the length of the River (sum).
Vector Selection and Analysis – Summits
Have the students load a point dataset of mountain summits with an attribute value for elevation.
Open the attribute column and show how you can sort a column by clicking on the column header.
Ask the students to make a new memory layer of the 10 highest summits. See if they can do it and then talk about the different ways to accomplish this.
Use this opportunity to load the matching DEM of the area and use the identify tool to show that the value (meters) of the summits point dataset matches the pixel value of the DEM. Remind them of the difference between raster and vector data.
Vector Selection and Analysis – Fire Perimeters
Load a Fire Perimeter dataset. Talk about dataset scale and how this dataset contains overlapping polygons representing every day of a fire. (go into layer styling and reduce the opacity to show the overlaps). Talk about temporal data.
Ask the students how to isolate the LAST polygon, representing the final extent of a fire. (attribute table)
Have the students select by that attribute, paste as a new memory layer
Have the students calculate the statistics on the memory layer and ask them which statistic will tell us the total area burned in a given year.
Data Editing – Review
Review the data editing fundamentals that were learned on Day 2.
Have the students create a new .shp layer (not a memory layer). Talk about the difference between disk layers and memory layers. Have them create the new shapefile in an organized /data folder. Have them add a ‘Name’ attribute and talk about attribute field types (string, integer, double)
Load in a WMS imagery layer
Have the students digitize some building footprints in their town.
Save the edits, look at the timestamps on the file on the disk as they do this.
Save the qgis project and reinforce the difference between the qgis project file and the linked data within a qgis project file.
Student Project Work
Now I had the students start to work on their individual projects which they identified on day 2.
This involved lots of one-on-one time working through issues for each project and ran the gamut of all sorts of concepts
I had a access to a smartboard, so I had all of the students gather around (away from their computers) to go over the concepts from Day 1 in a group format. I asked them answer questions about menus and specific purposes of certain functions.
Map Projections
Show a globe and talk about a round-ish earth and what has to happen to made a 2D map of a 3d object.
Talk about how different projections preserve different attributes and are designed for different extents.
Talk about units and how units are associated with a projection/CRS
Zoom in and out with google maps to show the transition from a sphere to a 2d Pseudo-Mercator projection.
Load the 50m_admin_0_countries into qgis. Talk about the ‘on-th-fly’ default and how datasets have a CRS, and the map view can have its own. Lots of math happening behind the scenes…
Set the map view to EPSG: 3857 and talk about disortion. Note the size of Greenland in relation to South America
Set the map view to EPSG: 8857 (Equal Earth) and talk area being preserved. Note the size of Greenland in relation to South America
Choose a few other projections and reproject the map view. Use EPSG codes via the qgis search dialogue in order to alleviate errors at this stage. Choose a polar projection and see the data ‘break’
Getting Data from the Internet
Talk about getting data from the internet and how it will come in various levels of quality, extents, scales and projections.
Adding a WMS basemap layer to QGIS: http://gis.sinica.edu.tw/worldmap/wmts/1.0.0/WMTSCapabilities.xml
Add in the imagery layer and zoom around. this is a crowd-pleaser
Have the students unzip it in the downloads folder, look at the file types, copy it to their Users/Username/data folder and highlight the importance of keeping it organized. Use this oportunity to talk again about data types (.shp, .geodb, .gpkg, .csv)
Geoprocessing Fundamentals
Buffers: load a polyline layer (I chose a trails dataset), and create a half-mile buffer dataset. Calculate area of the output polygons via Vector>Analysis>Basic Statistics…
Clip: Select a county from the Montana counties dataset, and clip the Montana geology dataset to the the county boundary. Use this opportunity to style the layer, label it, and ask questions about the geology polygons, where are the sedimentary types, where are the igneous?
Editing: Introduce the editor toolbar, memory layers and basic editing tools. Make a rectangle and clip the geology layer to the rectangle. Edit some nodes on an existing dataset. Delete by selection tool, Delete by row selection in attribute table, select by expression, delete selected features. Reinforce the concept of a memory layer vs. saving data to disk.
Symbolization
Graduated Symbols: Load ne_10m_states_provinces, ne_10m_populated_places, view the attibute table and have the students locate the field for population on the pop_places dataset.
Use the styling assistant to setuip the graduated symbols:
Layer_styling>simple marker>Size>menu>Assistant…
Source: Choose POP_MAX field
Refresh
Set output sizes from 1 to 10
Collect Field Data
Talk about Field Data collection and the three main geometry types (points, lines, and polygons)
If the students have smartphones, have them install the Input app and have them all login to the same username and password that you have previously setup.
Using the default project, or one that you have setup with that account, head out into the field and have them all collect data, points, lines, and polygons and take some geotagged pictures as well. Have them all sync the data to the public mergin cloud.
Back in the lab, have all of the students install the Mergin plugin via the QGIS Plugins menu.
Have them all login to the same account, load the project locally, and then sync all of the data.
Explore the data that they have collected, look at the attribute tables, and show them how a geotagged photo has the filename in the attribute table and the photo with the matching filename is also stored in the mergin project folder.
On this night, I had to source some custom datasets from the internet that each student would need to complete their project. Give them links if possible so that the student has to download, unzip and organize the data into their project.
Print Composer
Introduce the Print Composer and explain that it is a different window, but linked to the main qgis window.
New Print Layout>Name it for the paper size
Right-Click on the page>Page Properties>Set US Letter Page Size
Add Map
Move tool (layout manipulation)
Move Content Tool (set map extent and scale) #make sure they understand this tool difference
Scale, North Arrow, Title, Legend, Shapes, Attribute Table
Have the students style and print a 8.5×11″ map to the color printer using a selection of the datasets that they have active in qgis (pop_places, geology, etc.)
Setup Workspace panels so everyone has the same view #I made them all use the format that I use myself, that is Layers, Browser, Layer Styling, and Processing Toolbox all docked together on the left, tabs at the bottom:
Add Data from Browser tab. Have the NaturalEarthData full geopackage loaded on each machine. Talk about the concept of a file on the hard drive and how the QGIS browser is just an index (link) to the actual file. This is assuming that this has been preloaded into each student’s user folder
Styling features (live from the Layer Styling tab)
Accessing attribute tables
Filtering features in a layer
Zooming to a layer
Select Tool (Drag a box, Single Select, Select by attribute rows)
Copy selected features
Paste features as a temporary scratch layer (talk about the concept of a memory layer and how it related to geoprocessing fundamentals)
Labeling Features (lookup attribute table column to connect the concept)
Raster Versus Vector
To explain the difference between raster and vector, load the USDA plant hardiness geotiff and the vectorized plant hardiness map to see the difference between the two types of data
Talk about pixels (jpeg/low-res image example) and vector (points creating shapes with math to create objects)
Why use a raster dataset? #continuous data, more detail, each cell has a value
Why use a vector dataset? #generalized (grouped) values, easier to use with many geoprocessing tools
Raster Processing – Elevation
Now that the concept of a raster dataset is established, load a 1 arc second NED DEM, (projected to UTM zone in meters)
Using the Identify Tool, click on a valley to read the elevation and then a peak to read the elevation #reinforce raster concepts
Create a contour dataset (Processing Toolbox>GDAL>Raster Extraction>Contour) #talk about how vector datasets can be created from raster datasets
Create a hillshade (Processing Toolbox>GDAL>Raster Analysis>Hillshade) #talk about how vector datasets can be created from raster datasets
Joining Attribute Data
To introduce students to the concept of a join, we will join some US vehicle crash fataility data to the ne_10m_states_provinces dataset.
First, talk about the structure of a shapefile on a hard drive, (.shp, .dbf, and .shx) note the purpose of each file type
Open a .csv of the crash data in notepad and talk about what a .csv is–‘flat’ data. Note that the commas create columns
Open the attribute table for the ne_10m_states_provinces dataset and have the students identify a column that matches in the .csv
Have the students add the .csv from the qgis browser. Reinforce the concept of ‘flat’ non-spatial data denoted by the icon. nothing draws in the map view.
Have the students create the join
Then look at the attribute table to see the joined columns
Have the students symbolize the ne_10m_states_provinces dataset using the Deaths-per-100000-population dataset. Talk about Graduated symbolization, data ‘buckets’, and number of classes
Use a liner color ramp and have the students interpret it and ask then what they notice about the map they just made. Why are there low numbers in the Northeast even though there are far more people and cars than many of the other states?
TLDR: Though there are immense hurdles these days in terms of devices and media to distract them, the kids are alright.
About a week ago I taught an intro to qgis class through the Coding For Kids program in Stevensville, Montana. As a mid-30s guy with exactly zero kids and pretty much zero contact with kids of this age range, the biggest question on my mind going into it was where to start and what level of general computer and complex application literacy they would have. I spent my high school years building PC’s–tearing them apart to install video cards and LANing them together with my friends so we could play countersrike without 56k modem lag. Needless to say, things are different in 2021–would I be teaching them how to unzip files and navigate a file browser, or would we all be happily learning about different geoprocessing tools by day two? I mean, the kids and their tik tok these days, amirite?
So there I was, days before the class started, hemming and hawing about where to start the curriculum so as to not be boring but not overwhelm the students too early general computer literacy was a roadblock. With 6 students ranging from 8th grade to a graduated senior, I was at a loss.
Monday morning came and it was time to get cracking! The 6 students all seemed excited to be there and energized about learning something new. We started off with a few classic icebreakers and intros, talked about maps and what they knew about maps in general, but pretty quickly we got down to business and got going with QGIS. The students exceeded all expectations that I had and picked up some of the fundamental concepts pretty easily and by and large didn’t have any of the computer/file-management knowledge that I was (irrationally) concerned about. We covered geoprocessing, styling maps, projections, joining data and so much more! I’ve added the lesson plans here for those that would like to see the material that was covered. The students worked through the material quite quickly and had me scrambling a little bit for new curriculum later in the week. Below is what we accomplished through 5 days of class, 5 hours each day (3 on the last day).
QGIS is amazing. I’ve been using qgis for a really long time and it used to have all kinds of stability and usability issues. Now, it is so stable, so user-friendly, that I use it professionally and it was also great for teaching.
Natural Earth Data is amazing. I don’ even know what the cartography community did before this was a thing. Oh waut,es I do, everyone spent hours scrambling around for random datasets to create basic maps that never quite aligned properly and it was a mess. This dataset is so good, so lean, that it is always a joy to use professionally as well as a teaching aid.
Maps are amazing – I love maps–always have. I live and breathe them and it is always exciting to see young people light up when they make connections to maps and the curtain for how they are made is pulled back. It was so rewarding to see various concepts click and for the students to take off down a tangent exploring the possibilities.
Lessons Learned
Phone Apps – I went to high school in a world of land lines and giant rolls of clear film with days of class work being pulled across an overhead projector. Smartphones and social media apps were not a thing, and for that I am grateful. The addiction to to social media apps is real, and phones were the most counter-productive force in my classroom. 5 of the 6 students would be sticking with me and following along, but one would be scrolling their phone, and then when they came back to reality, I would have to go over and individually catch them up. While I was doing this, the other students would pull their phones out during the downtime. Ugh. If I do this again, I will make a cell phone policy clear at the beginning and also be a little more hip on the ’20-minute bathroom break’, which I now see was just a cover for a phone scroll session. Need to go to the bathroom?–phone stays in the room.
Map Styling – I use Adobe Illustrator to create maps and use qgis largely for analysis and geoprocessing. I use basic styles, but really stick to the rudimentary stuff. I should have known this, but the students were really keen on styling the maps. I need to learn more about advanced styles and how to make that part more engaging. Also, some of the default style options in QGIS can be pretty horrid, so guiding them to some more reasonable choices would be a good start.
Lab Data Persistence – The lab that was graciously provided for the class was a public school lab that had software which wiped out the user folder each night. Microsoft ‘one-drive’ was available, but it required re-syncing data each morning and caused all sorts of linking issues with data and qgis project documents. Make sure that there is a stable place for students to store data, either a network share or a usb flash drive for each student. (I ended up setting up a flash drive for each student)
Conclusion
It was a fun week and I’m impressed with how the students did. If you are here because you’re also going to be teaching QGIS, let me know if any of this was useful!
The GNIS (Geographic Names Information System) is the authoritative* source of information for place names in the United States. It is a fantastic dataset and despite the fact that there seems to be no reasonable way to get errors fixed–namely grossly misplaced peaks and elevation values–it has proven to be a valuable resource for cartographers. All of the features in the GNIS dataset are point features even if they represent features that are distinctly linear.
One of the things that I love about the original USGS topo 1:24k maps and I lament about the new US Topo products is the lack of feature labels on bézier curves that are now printed as straight block labels instead. I know that there are funding issues, usability issues, etc., but nevertheless, I hate to see this art fade away. For smaller features this isn’t quite as much of an issue, but for larger features like mountain ranges, a single point label on a particular quad is poor at best and quite misleading in my opinion.
Railroad Canyon label from the original quad map versus US topo on the right. Long mountain ranges rendered on a bézier curve multiple times to accurately depict the feature (left) versus a single point text label to label the whole range (right).
SO! What are we as enterprising cartographers to do to resurrect these great labels and use them for our modern cartographic work? Well, that’s what this project is all about.
The Project
This project aims to tap into the skills, time, and expertise of the cartographic community to build a rich, multi-scale dataset of these linear features. Together we can all work on a particular part of the country and then combine our efforts into a single dataset of linear feature labels. It carries an open source license and will be available in the public domain for all to use.
I’ve downloaded the GNIS point dataset and isolated the feature types that I think would benefit from being represented as a linear feature instead of a point feature. Mountain ranges are the obvious example, but other feature types including valleys, canyons, plains, and gulches are included as well. I’ve converted these selected point features into linear features and retained all of the attributes.
The GNIS feature types included in the GNIS-Bézier projectA GNIS-bézier feature for the Crazy Mountains with attached attributes in Adobe Illustrator with MAPublisher.
Help!
If you’ve made it this far, maybe you are the type of person that wants to contribute to the GNIS-Bézier project. Great! Let’s make this dataset together and make it free for everyone so we can all have better maps in our lives for years to come!
Step 1. Choose an Area
The first step to contributing to the project is to choose an area to work on. I’ve broken the country up into blocks that will allow for a ~360 square kilometer area to be worked on at a time. These constraints are based on the maximum artboard size that Adobe Illustrator supports when this area is displayed at ~1:50,000. Use the map below to identify an unclaimed block (click to enlarge). Any of the blocks below that are NOT filled with blue are unclaimed and up for grabs. Send me an email to jamierob @ gmail dot com to claim one. The Video below will help explain the data behind the project and choosing a piece of the country for you to work on.
In progress map: Blue=taken, Yellow=available. Updated 10/04/2020
Project Overview
Step 2. Download the data for your block
After you have your block, it will take me a little while to prep the documents for you. I have a semi-automated procedure that stitches together all of the original quad maps, de-collars and re-projects them, and then strips out a lot of the colors except for the relevant ones to placing the labels. The topo images and the GNIS-Bézier features are then put into an Adobe Illustrator document and ready to go. Once that is done I’ll post the files to the github project page and you can either download it or check it out. Don’t let a lack of git knowledge keep you from pitching in! Here is the repository if you want to check out the work so far.
Step 3: Edit the features
Once you get the features from me, you’re ready to get your hands dirty and move some lines. You’ll need basic knowledge of Adobe Illustrator, file management, and an attention to detail. There are some custom path scripts that really help to refine the bézier curves. An extension for Adobe Illustrator called MAPublisher is also required, but if you don’t have one–that’s ok–you can get a 14-day trial from Avenza and hopefully that should be sufficient for you to complete your block. The video below explains the procedure for downloading and editing your block of data in Adobe Illustrator. The basic steps to remember are:
Edit the two nodes to make each line feature match the shape of the physical feature in question. Add new nodes as necessary but only if needed.
Attributes – When the line is finished, update the ‘workstatus’ attribute for the feature as follows:
x – The edit is complete and no other work is needed. It falls completely within the block being edited.
q – you have a question about the feature. Someone else should take a look at it.
e – The feature goes off of the edge of the block you are editing. When this feature is merged into the nation-wide document, this will need to be edited/fixed.
Delete – If you think that the feature is just too small or silly to warrant an actual linear feature label, just delete the feature.
Remember: These are very small illustrator files from a size perspective because they do not have embedded images or saved with PDF compatibility. If you try to save it with PDF compatibility checked or embed the images, it will probably choke.
Editing and Contributing
Step 4: Submit your work back to the project
When you’re finished editing your block, either send the .ai file back to me or make a pull request and commit it via git. I’ll combine it into the full usa .ai document and we’ll slowly build up the dataset for the whole country. If/when all of the blocks are completed, we’ll have a single .ai document with bézier lines, with attributes for the whole country. We can assign map scale to features based on the line length and/or if the label text will fit on the line at a given scale. I haven’t quite sorted that part out yet. Finally, the dataset can then be exported out to other geodata formats so it can be used in other applications. The reason for starting in Illustrator with MAPublisher is to have the clean, bézier features first because when the features are exported (likely to shapefile), nodes will be added along the curves.
Thank you for reading and please consider working on a block if you’re interested! If you have any questions please post them below or send me an email.
*authoritative from a European colonization standpoint. The GNIS dataset contains place names from after Europeans arrived on Turtle Island and created names for these features. Many indigenous place names are not represented and some current place names are racist. I fully support efforts to rename these features to provide long overdue respect and recognition of the original names that were attached to these features.
Back in 2017 I gave a presentation about a set of custom scripts and shortcuts for enhanced productivity in Adobe Illustrator. They have proven indispensable for me and my cartographic work. If you want to spend less time in menus and also have some special tools to help you make maps, check out the video below. There are also installation videos for Mac and Windows. I’ve updated the shortcuts and scripts for Illustrator 2020 on Mac and Windows. Download the scripts and shortcuts here.
Back in 2017, I gave a presentation at NACIS about a selection of scripts and shortcuts for enhanced productivity when making maps in Adobe Illustrator. I’ve updated the repository and the shortcuts for Illustrator 2020. Below are two videos to show the process of setting them on Windows 10 and MacOS Mojave.
