FME 2011 Sneak Peek: PostGIS

Hi all,
You might think that FME doesn’t have much to do with open source software, but, as Michael Weisman once wrote, FME is an excellent “bridge between the open source and proprietary gis worlds”.

So with the FOSS4G “conference” (for Open Source Geospatial Software) about to start in Barcelona, Spain, it seemed a good time to open my FME2011 coverage with an item about what updates and enhancements we’ve made specifically for PostGIS in our upcoming release.

As it turns out, there are rather a lot. In this post we’ll cover:

• Geodetic Data
• Unicode
• Measures
• Geometry model enhancements

Geodetic Data Support

A new GEOGRAPHY field type introduced in PostGIS 1.5 allows native support for data in longitude/latitude coordinates in PostGIS. Of course, reading and writing such columns is also implemented within FME.

The first inkling you’ll get of this in FME is that the parameter formerly known as “geometry column” was renamed to “spatial column”, since we now support both geometry and geography types, and a new parameter is added to select what type of features you wish to write (below):

On running the workspace the first thing I get is an error:

That’s because the source data was non-Lat/Long, so FME set up the writer to match; and a non Lat/Long coordinate system is incompatible with a Georgraphy column. A simple change of the Coordinate System parameter to EPSG#4326 and we’re in business! I can write the data and read it back, for example with the FME Universal Viewer (below):

Incidentally, if you use Geography columns then don’t forget about dynamic coordinate systems in FME (_AZMEA_ and _AZMED_) and how useful they can be.

Unicode Support

Maybe it’s because I’m English that I have a hard time understanding language encoding – or maybe everyone does but hides it well. Either way, I always use this Joel Spolsky article when I have to brush up on unicode. I think it’s worth a read; but if you are just concerned with what – rather than how – then skip to the foot of this section for the basics…

So, when we say that FME now supports Unicode for PostGIS, what do we mean?

Well, if you read the above article, you’ll know that it’s not just the ‘code point’ (the Unicode ID number) of a character that matters, but how it is stored (the encoding). FME now transfers data from the PostGIS reader and to the PostGIS writer using UTF8 encoding. UTF8 is good because it is variable length and can handle any Unicode character. You could say that it is the FFS format of character encoding!

It doesn’t matter if your destination database is set up to use a different encoding. FME can hand it off to PostGIS as UTF8, and PostGIS will take it from there.

So, you can read non-English characters into FME (or create it, like this Greek string in an AttributeSetter):

And FME will happily write that data into PostGIS:

Oh – and to prove we read the info just as well, here it is read back into the FME Viewer:

The Logger transformer will even tell you the character encoding – in this case UTF8.

Interestingly, the string was only shown correctly when I opened the FME Options dialog and changed the log font to Tahoma. Neither Courier, nor Arial, nor Times Roman supported the necessary characters; showing Unicode issues can arise in the most unexpected places.

As a user, this means that you can merely send non-English characters to PostGIS, and/or read them back, and all will be well. The data will not turn into a row of question marks. You don’t even need to force your computer into a different locale to make it work.

One other thing you can do is create attributes and tables with non-English names. However, to do that you will need to set your computer to a different locale, else Workbench won’t save them in your workspace file.

Measure Support

Personally I like using measures, and so it was pleasing to see that FME2011 will provide support for reading and writing these to PostGIS.

For the uninitiated, measures are a form of linear referencing; i.e. they specify the distance along a line that a certain feature or event occurs. For example, a bus route is 10km long; the bus makes stops at 1.1km, 2.4km, 4.1km, etc. The bus route is the line feature, the stops are the events, and the 1.1, 2.4, 4.1… are the measures recording at which point along the route the event occurs.

For a while the FME geometry model has had the ability to store measures, and Workbench the ability to manipulate them with various transformers. So as a test I created this (rather elaborate) workspace:

What it does is read a transit line in Vancouver, add in the stations as vertices, and tag the line with a measure where zero (0) means this vertex is not a station and any positive number is a station ID.

This rather large screenshot (click to enlarge)…

…shows a highlighted vertex representing station #10 (see the measure value). The location has been highlighted on the feature itself. So we can not only write measures to PostGIS, but read existing ones back.

As a quick tip, to get that info back into a set of station features, I would read the PostGIS database, use the Chopper transformer to chop the line back into points, the MeasureExtractor to get the ID number, then a Tester to throw away features where ID=0 (i.e. not a station).

Geometry Support Updates

FME2011 includes some useful updates to its PostGIS geometry support, and in particular changes the default setting for one parameter about which it is useful to know. So here’s the full scoop…

Perhaps the first thing to notice is a parameter which you set when adding a PostGIS writer (or creating a new workspace containing a PostGIS writer). The parameter is labelled “Create Generic Spatial Columns”

This parameter basically creates PostGIS tables without a specific geometry type; i.e. your FME feature types have their Allowed Geometry setting greyed out and unusable:

If you turn off the parameter then you can set a specific geometry type for each table:

The important point is that this parameter used to default to no, but now defaults to yes! We figure it’s more user-friendly in the long run, but existing users might not notice the change.

Another update for 2011 is that, when you do turn off the parameter, you’ll notice the list of allowed geometries is much more complete than it was in FME2010.

PostGIS has had support for curves since v1.4, and you’ll notice that 2011 has introduced FME support for these. In fact we now support circular strings, multicurves, compound curves, and curve polygons. The other revision to this list was that we split up geometries and renamed them to match the true PostGIS list. The list might be a little longer, but it should be easier for the user to figure out what to select if it matches what they are familiar with in PostGIS.

Minor Fixes

Along with all those major updates there are – as always – several minor fixes and updates. So even if you don’t need any of the functionality mentioned above, we recommend you upgrade to the latest FME version as soon as you are able.

FME2011 is available in beta form right now, and scheduled for full release in January 2011.

Wrapup

So there you have it. FME2011 will introduce many new reasons to use it in conjunction with PostGIS, and help cement FME’s reputation as the proprietary solution for smoothing out the bumps in your open source GIS solution.

Thanks: Thanks to Raven and Paul in Safe’s database team for their help in putting this article together (and kudos to the same for putting together such a great set of functionality).

Mark Ireland

Mark, aka iMark, is the FME Evangelist (est. 2004) and has a passion for FME Training. He likes being able to help people understand and use technology in new and interesting ways. One of his other passions is football (aka. Soccer). He likes both technology and soccer so much that he wrote an article about the two together! Who would’ve thought? (Answer: iMark)