diana maps

After DirectionMag hosted Esri’s free webinar, ArcGIS Notebooks – Using GIS and Python Notebooks for Workflow Automation last year, a number of attendees followed up with us, many admitting that they lacked the skills to do the kind of work presented. Some wondered whether it is really important to have programming skills in today’s GIS environment; others wondered how they could get the skills they needed quickly and inexpensively. In today’s post, I want to address both of these questions.

How important are programming skills in today’s GIS environment?

The answer depends on how badly you want the job.

 “…the skills set of programming together with a capacity to problem-solve is a particularly powerful and pertinent combination for those eager to procure—and maintain—GIS employment.”

—Diana Sinton, “GIS Jobs of Today: How does ‘Open’ Fit into The Mix?,” 2016

 Over four years ago, Diana Sinton, DirectionsMag columnist and the executive director of the University Consortium for Geographic Information Science, wrote a series of columns on the requirements of GIS jobs. Even then, there was consensus on the importance of programming skills. 

 “Without a doubt, GIS professional with Python and other programming language skills will get the best positions. Those with programming expertise garner higher salaries and are more valuable to employers.”

—John Gabriel, co-owner of Alsea Geospatial Inc., from “GIS Jobs of Today: Should you have programming skills?” by Diana Sinton, 2016.

Taking a quick look through the GIS job ads in 2021, there is little doubt that programming is highly valued by hiring managers. So, the question isn’t if you need the skills, but rather, why have you hestitated?

Python was chosen by Esri for ArcGIS specifically because it is easy to learn, while also being scalable and complex enough for experts. There are literally hundreds of online resources for learning it, so there is no more room for excuses. It’s time to get to work!

 How can I learn Python as quickly and inexpensively as possible?

There are two schools of thought on this one:

1. You can learn Python within the context of ArcGIS, and rely heavily on Esri tutorials, or

2. You can learn open-source Python coding, then tweak your knowledge to use Python within ArcGIS.

  If ArcGIS is your primary concern, a good place to start is with Esri’s free, interactive webcourse, “Python for Everyone.” The course takes about 3 hours to complete.

Then hop back into LinkedIn and sign up for Jennifer Harrison’s 2.5 hour course, Learning ArcGIS Python Scripting.

These courses, along with a copy of the book, “Python Scripting for ArcGIS Pro” by Paul A. Zandbergen, will give you a solid foundation of skills. (DirectionsMag ran a review of the first edition of this book in 2013. To get an idea of how effective the book will be, you may be interested in reading the reviews before picking up a copy.)

Finally, reinforce your skills and get a look at their larger context, with Penn State’s Geog 485: GIS Programming and Software Development course.

 If you’d rather learn Python generally, and dive into ArcGIS later, take a look at Google’s Python Class, UDACITY’s Introduction to Python, and this free online tutorial from the University of Chicago to get your feet wet.

With the basics behind you, consider moving into some of the low-cost training courses available from online academies; Penn State’s Geog course material recommends the beginner-friendly courses offered by codeacademy.

If you are still looking for more, the wiki “Python for Non-Programmers” provides a huge list of resources to explore.

After working your way through these resources, you’ll be feeling pretty comfortable with Python, but heed this word of warning. Head over to DirectionsMag for this quick read before you dive into ArcGIS: “10 Common Python Errors of Beginning ArcGIS Programmers.”

You’ll be more than ready for our next Python-related webinar!

Reposted from The DirectionsMag Geospatial Community Blog, an extension of Directions Magazine. Visit us for daily geospatial news, exclusive articles, geospatial webinars, and podcasts. If you are interested in contributing, please email editors@directionsmag.com.  https://www.linkedin.com/pulse/yes-you-need-python-skills-heres-how-get-them-rebeckah-flowers/

Directions Magazine’s Diana Sinton sat down with Dr. Jill Saligoe-Simmel, the product manager of SDI and INSPIRE for Esri, to discuss the increasingly clear role that spatial data infrastructure plays in problem-solving.

Diana Sinton: Your position at Esri is a new one. What has been driving the increased interest in, and awareness of, spatial data infrastructures?

Dr. Saligoe-Simmel: Whether it’s called an NSDI, OneMap, Regional Information System, GeoPortal (as in Africa GeoPortal), or otherwise, spatial data infrastructures (SDIs) continue to be relevant and pervasive at all levels of government and across many sectors. At their finest, they enable cross-border and multi-organizational collaboration to address our most significant social and environmental challenges, including natural and human-made disasters. These collaborations promote efficiency at all levels of government.

In the past, metadata catalogs and data exchange dominated SDI initiatives, often with a “build it and they will come” approach. Over the years, with a few notable exceptions, many SDIs have struggled to gain traction. Yet the overall objectives of SDI are quite sound, and may be more relevant than ever. That’s why I’m optimistic about seeing new patterns emerging that expand the focus of SDI on problem-solving.

Today, SDIs are rapidly evolving. Together, the internet and cloud computing are transforming the way organizations manage data and collaborate. Web GIS is significantly easier to use, deploy, and integrate into an SDI ecosystem than traditional systems. For example, portals manage all aspects of geographic information, including data and services, maps, analytical models, applications, workflows, and even data security and access. Organizations are connected through a network of portals. Data service APIs enable users to bring together data dynamically from distributed systems into a host of applications.

Through this emerging geospatial infrastructure, users can now inexpensively and efficiently access immense amounts of geographic data. Community members can easily share maps, analytical models, applications, and workflows across and among multiple users and organizations. The result is a secure but highly collaborative environment — an integrated set of deployments focused on shared goals.

For over 30 years, Esri has promoted and supported SDIs at local, state, regional, national, and global scales. We have deep experience in SDI, interoperability, and open standards. It’s an opportune time to make sure we are supporting SDI success in this new era of geospatial infrastructure.

Diana Sinton: Now that you’ve settled into this new position, towards what types of goals are you working? What is a typical day for you? 

Dr. Saligoe-Simmel: As a product manager, I focus on identifying and communicating SDI-related patterns and practices, and how we can best support them with our underlying platform. While listening to our customers and the market, I also collect, quantify, and prioritize common problems where we can help evolving SDI to be more successful. Each day, I spend time listening to and observing the SDI marketplace and thinking strategically about how we can further our customer’s success. By understanding our customers’ experiences, we can identify common patterns among successful SDI and share those patterns and practices with others.

In her book, Governing the Commons (1990), Elinore Ostrom recognizes that “common pool resources” operate under diverse institutional arrangements. This certainly applies to SDIs, where there isn’t a singular organizational model for success. Cooperation can take many forms, and common patterns emerge — for example, a meaningful pattern is: Framework data are open and clearly identified as authoritative. When users find themselves awash in a sea of data, the open data movement is a victim of its success. Are the data the best available? Are they sourced from the local authority? Authoritative data provide a single point of reference for all parties to work from — a common operating picture — that is critical to effective government operations.

NC OneMap is an authoritative discovery point of geospatial data for the state comprised of data supplied by multiple agencies. Therefore, it was important for them to establish a set of standards for documenting resources so that authoritative data providers do it consistently and in a standardized fashion. This way, when someone performs a search, they can be assured that results list all the relevant resources. NC OneMap shares its best practices for sharing authoritative data on their Open Data site. 

The need to manage authoritative data effectively will grow in importance as open data policies, such as the U.S. OPEN Government Data Act and the EU Open Data Directive, are enacted.

Diana Sinton: The more front-facing piece of spatial decision infrastructure is readily accessible data, but much has to happen for that to happen effectively and efficiently. Do you have any examples or stories about those essential, but less visible, parts of SDI (problem-solving, reducing duplication and effort, etc.)?

Dr. Saligoe-Simmel: The most significant challenges for most SDI remain policy and organizational, not technology. Though SDI communities have laid the groundwork for governance and policies, there is still much to be done.

For example, SDI’s have traditionally focused on the roles and responsibilities of their coordinating bodies and contributing partners. Over the last several years, we observe that the most successful SDIs are often mission-focused – sometimes encompassing the missions of multiple stakeholders. Communities of practice can amplify the value of their SDI by designing for end-use by the consumers (who are sometimes, also, partners).

Around the world, people and organizations aspire to be safe, healthy, prosperous, and more. These aspirations can be translated into initiatives that citizens understand and support. The use-cases (or value proposition) drive the SDI data. In a 2014 Report of Stakeholder Engagement on Four Geospatial Issues with National Importance, National States Geographic Information Council (NSGIC) members recognized that mission-specific programs are typically successful, although not always in a way that meets the needs of the entire geospatial community. It remains that the positive impacts of mission-specific projects should shape our SDI thinking as we move forward.

A great example is Open Data DC, which shares hundreds of datasets freely among agencies, federal and regional governments, and with the public. What’s innovative is how they wrap their open data site in apps and data stories, such as preserving tree canopy and recycling, that engage their stakeholders and the public. Initiatives can also be a mechanism that facilitates framework data. For example, Addressing In DC shares information on the history and standards behind their comprehensive Master Address Repository (MAR) and provides educational information to guide people to use the data, apps, and map service APIs.

At a national level, HIFLD Open Data provides access to hundreds of national foundation-level geospatial data within the open public domain for use by local, state, federal, tribal, private sector, and community partners for community preparedness, resiliency, research, and more. It demonstrates a distributed portal model in action. In 2017, the U.S. Department of Homeland Security (DHS) launched dedicated websites to unify the response and recovery data aggregation efforts for hurricanes Harvey and Irma. Built on top of the HIFLD Open Data portal, these sites provided event-specific hubs to disseminate data and support the massive mapping activities for response and recovery.

Globally, we see similar patterns with the UN Sustainable Development Goals (SDG) Hub. Beyond data, GIS professionals, developers, and domain experts are collaborating with shared workflows, models, analyses, templates, and code. The next decade should prove an exciting time for SDIs, enabled by open platforms and the geospatial cloud.

Digital technologies have inspired and launched revolutions within the field of cartography in the recent past, bringing mobile mapmaking to our finger tips. Cartographers with an eye to the historic aesthetic are pursuing approaches to make new maps look old. But what about new ways to look at old maps? How are modern technologies being used to support research and investigations?

One way to appreciate historic cartographic treasures is to gift yourself new lenses with which to look. The Beinecke Library at Yale University has facilitated this process with the 1491 Martellus map in its collections. Produced by Henricus Martellus in the late 15^th century during the epoch of global explorations, contemporary scholars have long wondered about the knowledge that the cartographer included in his production. Unfortunately, the map’s faded and deteriorated condition has greatly obscured almost all of its text, at least to the naked eye. What is there that we just can’t see anymore? 

That’s the type of question that fuels The Lazarus Project, a research endeavor now based at the University of Rochester and directed by Gregory Heyworth. By using camera sensors that can capture light from wave lengths both smaller (ultraviolet) and larger (infrared) than the human eye can detect, Heyworth and his colleagues combine bands to produce new multispectral images of early manuscripts and other historical documents that allow us to see what had been otherwise obscured for centuries. Cartographic historian Chet Van Duzer oversees the Lazarus missions that deal with maps and globes, and he has worked closely with Roger Easton, an imaging scientist at the Rochester Institute of Technology, to produce the enhanced images of the Martellus Map, with funding from the National Endowment for the Humanities. Remember that children’s story about Harry the Dirty Dog, who refused so stubbornly to take a bath that eventually his family fails to recognize him until after a good scrubbing? Viewing the before-and-after examples of these multispectral images signifies a similarly magical visual transformation. 

 An image of Africa on the Martellus Map as it appears in natural light (left) and an image of the same section that has undergone multispectral image processing (right). (Images by Lazarus Project/MegaVision/RIT/EMEL, courtesy of the Beinecke Rare Book and Manuscript Library.) 

A particularly enticing feature of the Martellus map that was revealed was its descriptive texts. These proved challenging to uncover because Martellus had used different pigments for different texts, the same way we would choose a different color or fonts. Van Duzer recalls how Easton spent months tweaking the combinations of bands iteratively and uniquely to optimize the legibility one section of the map at a time. This labor of love was necessary to achieve the highest clarity and legibility possible, and Van Duzer has detailed his research findings in a new book, Henricus Martellus’s World Map at Yale (c. 1491). 

Designing the multispectral camera system to be a mobile and transportable one means that historical objects that would otherwise never travel— such as a 500+ year-old map — can undergo data capture under safely controlled conditions in their home institutions. An even older map on which the Lazarus scholars, particularly Ph.D. candidate Helen Davies, have been working is a 13^th century mappamundi in Vercelli, Italy. Previous attempts to clean and restore the map damaged it significantly, but the multispectral techniques are non-invasive and non-destructive. This is the only feasible way to approach such precious and unique items with the aim to generate clear views of the original content.

Gregory Heyworth working in Yale University’s Beinecke Library, with the Martellus map on an easel set in front of the camera. (Image courtesy of Chet Van Duzer.)

Text and images that have been compromised because of fading and material deterioration are one reason for lost research and educational opportunities. Other reasons include restricted access conditions for the protection of the items, or their physical characteristics; massively large maps are difficult or impossible to see in their entirety from one vantage point.

Digital versions address many of these concerns in a way that also enables greater access to the objects themselves. In the world of historical maps, the David Rumsey Collection at Stanford University is legendary. This vast collection of paper maps has been digitally scanned at such a high resolution that extreme viewing of the details is made possible at a greater level than is even possible to the naked eye (especially for those of us with eyesight compromised by age!).

Some of the maps in the Rumsey Collection are difficult to appreciate fully as scanned items, however, not because of their resolution, but because of their great size. For example, in the late 16^th century, an Italian nobleman named Urbano Monte became fascinated with the idea of designing and producing a large-format world map in the round, one that would be about 10 feet in diameter when assembled. How was it printed in preparation for this? The individual “tiles” were 60 sheets within a manuscript atlas that could be unbound, cut-out, and edge-matched. His intent by design was that the 60 map sheets would be assembled into a flat circular map that would be backed with wood, then made rotatable about its central point, like a big wagon wheel. It is unknown how many of these Monte maps were actually produced, and it’s unlikely that any were ever assembled as envisioned.

Rumsey owns one of the Monte manuscripts, printed in 1587, and has provided the world with high-resolution versions of its individual sheets. Monte’s beautiful and captivating map has now been re-projected as a globe as one of seven world and celestial maps that are available in an augmented reality format through the AR Globe app (free; currently available only in iOS).  Monte was imagining that a 10-foot diameter wooden disk that rotated would be a fascinating representation to explore the world, but he never had the chance to play with a mobile digital device! Being able to zoom and rotate his map – and the others that the app makes available is an exciting tool for exploration and discovery.

The 1587 Monte world map as a digital globe in virtual space (left) and a zoomed-in section of interest (right). Image source: Diana Sinton.

Not only are these maps aesthetically remarkable, but these digital formats enable unprecedented access for scholars. For example, Van Duzer has been exploring how Monte created his huge map — what sources he used for its geographical contents and images, and what inspired his map’s projection and rotatability. The number of historical maps may be a finite set, but there will increasingly be novel ways to consider these with new approaches and lenses.