23/07/2012

* Did you use the right "WGS84" ?


Did you use the right "WGS84" ?

The deeper I go into coordinate reference system (CRS) the more I start to question whether I am using the right one because accuracy is the name of the game when it concerns GIS data. Goggling here and there, I realized Malaysia itself uses 4 UTM Zone reference. If a map was prepared using the familiar WGS84 with the Authority ID:4326, things would be simple but what happened if the required map prepared by a third party set on finer specifications, how now? If the user who receives that map was unaware that the CRS was not really WGS84 but a combination of WGS84 and UTM zone, then, the user will never get the CRS accurately configured. Don't believe me, look up the CRS under the CRS Selector and find:

WGS84                         - Authority ID:4326
WGS84/ UTM zone 47N - Authority ID:32647
WGS84/ UTM zone 48N - Authority ID:32648
WGS84/ UTM zone 49N - Authority ID:32649
WGS84/ UTM zone 50N - Authority ID:32650

From the Internet UTM image I found, (I may be slightly off), that certain states overlap with the CRS that concerns UTM zones so this makes it more complicated:

WGS84/ UTM zone 47N - Perlis, Kedah, Penang, Perak, Selangor, Pahang, Negeri Sembilan, Malacca
WGS84/ UTM zone 48N - Kelantan, Terengganu, Pahang,Negeri Sembilan, Malacca, Johore
WGS84/ UTM zone 49N - Sarawak
WGS84/ UTM zone 50N - Sabah, Sarawak

You will start to appreciate this information when if you are required to convert GPS data from long-lat references to GIS references because the right CRS is crucial if you want to get your coordinates right. When I contacted JUPEM, I was told the CRS they use is either RSO or Cassini Soldner, all this makes it more challenging. I'm still muddling through this subject which is not my field but the purview of surveyors.

23 comments:

  1. Hi, Pak Abbas!

    Very interesting post! This was something I've wanted to discuss with you if we get to meet up when my wife and I return to Malaysia in lae September. (My wife and I finally got a Malaysia My Second Home (MM2H) visa in June!) We will be in KL through May or so of 2013 this time.

    One of the things I found when I was doing the "Township GIS" project was that the GDAL utilities *which QGIS uses for projecting and re-projecting layers) is pretty useless when dealing with the RSO projections (like the Michigan GeoRef projection used by our state government here). Hence I had to find a work-around (using uDig) for converting shapefiles from GeoRef to something that QGIS could handle.

    This also implies that QGIS would not be able to correctly handle shapefiles that Jupem might make available to users in Malaysia. One of the things I want to do is to see if the uDig workaround (https://sites.google.com/site/townshipgis/resources/problems-and-workarounds/converting-michigan-georef-to-state-plane-using-open-source-tools) would work with files from Jupem (or from anybody else using RSO).

    There have been posts by Malaysians in the GDAL forums begging the folks there to do something about the RSO problem, as this makes QGIS (and other open source software that depends on GDAL utilities) useless for work involving RSO-projected files. I don't know if the GDAL community will be able to act on this soon, because their foremost programmer has gone to work for Google, and who knows if he will have the time to devote to the GDAL tools like he used to?

    Anyway, I'm not an expert on map projections; my only background being a class I took when I was working on my MGIS program. But I'm interesting in the subject, especially since the RSO problem nearly derailed my Township GIS project!

    Keep up the wonderful posts and the good work you are doing!

    Cheers!

    Howard Yamaguchi

    ReplyDelete
  2. Yamaguchi-san

    I will not be surprised there are areas where QGIS is still weak, after all, Open Source software is very much dependent on contribution from the OS community. Next, it is still Version 1., even OpenOffice.org only started to bloom after Ver. 3.0. I am already thankful that a software such as QGIS existed in the first place even with all its shortcoming simply because it is free and that helps to kickstart many an agency that simply cannot afford a GIS software but wants to develop its own GIS database. If QGIS with its current specs cannot deliver what one wants and someone wants something fast now, I see no other option but to cross over into proprietary GIS territory.

    ReplyDelete
  3. Pak Abbas,

    One of the characteristics of the open source community is that when one tool doesn't quite do what they want, they tinker with another one, and another one, until they get what they want. Who knows, one of the other open source GIS tools, other than uDig, will be able to handle RSO-projected files! But yes, when all other open source tools don't do the job, it's time to break out the wallet!

    ReplyDelete
    Replies
    1. >But yes, when all other open source tools don't do the job, it's time to break out the wallet!

      Ha-ha-ha-ha yes, that's reality. Beggars cannot be choosers

      Delete
  4. The UTM zone (Z = UTM zone number) can be calculated from the geodetic longitude of the point (converted to decimal degrees):
    - Z = (180 + λ) / 6 (east longitude)
    - Z = (180 – λ) / 6 (west longitude)
    If the computed zone value Z results in a decimal quantity, then the zone must be incremented by one whole zone number.

    ReplyDelete
    Replies
    1. Wah...thank you, thank you for this information. Now, if only you can translate that in simple terms so a non-GIS user (like many top managers) can understand, I would be much obliged. Putting technical issues into layman words has always been a great challenge for me.

      Delete
  5. I think in most layman application like GPS navigation, google map, geocaching, we just use WGS84. Anyway if you need to reproject your map to another projection, you need to know the parameters. You can refer here for the parameters: http://www.spatialreference.org/

    ReplyDelete
    Replies
    1. But the many of the RSO definitions in spatialreference.org also derive from GDAL. So please use the RSO-related EPSG references very carefully --- the chances are, they won't work very well, if at all, in QGIS.

      Delete
    2. OK. In town planning, the use of state Cassini maps is more prevalent since planning work is mainly lot base. That affects all local planning authorities nationwide. The use of RSO is more often used for projects which cross state administrative boundaries such as macro and economic corridor planning studies which are not many but there are.

      Delete
  6. Nice write up. To my understanding, Coordinate Reference System (CRS, e.g. WGS84) is not the same as Spatial Reference System (SRS, e.g. UTM). Basically, CRS is only an ingredient to make a SRS, and not the SRS of itself. That means UTM uses WGS84 to make a projection..

    Maybe we can put it this way:
    CRS = A reference ellipsoid (a 3D ellipse) we use to pinpoint ourselves from each other. The most popular ones are WGS84 and GRS80.
    SRS = How we project those 3D ellipse into a 2D planar, like paper/monitor screen. There is no perfect way to make a flat surface from a 3D ellipse, so mathematical models are used to project CRS into a 2D planar. For instance, UTM system uses a secant transverse Mercator projection using WGS84.

    So, WGS84 data by itself is unprojected. It's still in the 3D ellipse format. If you display (draw) the map without specifying any kind of projection model, you're actually using the Plate Carrée projection to turn it into a 2D model (this projection basically treats longitude+latitude data from WGS84 and turn them into simple X and Y lines, which will make a VERY distorted map). WGS84 data that's in a different projection must simply be reprojected into the desired projection.

    The Authority ID (SRID)found on spatialreference.org is the reference number of a SRS. The most common authority is EPSG. There are others(like ESRI; heck-we can even make our own projection..), but just to be safe, use the ones authorized by EPSG (e.g. EPSG:3376 - it has 'EPSG' in front of the SRID number). As Howard commented above, some RSO data found in spatialreference.org won't work well with GDAL (e.g. SR-ORG:7174 - this is an RSO projection of West Malaysia, but not authorized by EPSG). If not, the GDAL tools won't work correctly because of the wrong SRID.. At least this is of my understanding..

    Other notes:
    - WGS84 (SRID: 4326) = WGS84 unprojected
    - WGS84 UTM 47N (SRID:32647) = WGS84 projected using Plate Carrée model
    - RSO = The old RSO uses an old Everest ellipsoid ref, projected using some kind of oblique Mercator model. I believe the newer RSO uses GRS80 as the ellipsoid ref..
    - Cassini = a simple non-perspective cylindrical projection
    - Google (SRID:3785) = uses a mercator projection

    There are many tools to convert projections, most can be found free online. But the essential step I believe is to understand what kind of data we already have.. In your case of data spans in multiple SRID (4xUTM SRIDs here in Malaysia), there are few options:
    1. Store everything in a database unprojected (WGS84), and transform on-the-fly with every query (need to know a bit of SQL to do this)
    2. Maintain everything in original UTM projections in your database, by partitioning the data using table inheritance (REALLY need to master your SQL for this)
    3. Choose a different kind of projection all together. Choose one that may fit ALL of your data. You can use the Google Mercator projection, which is fairly good. Or opt for the new projection adopted by JUPEM: GDM2000 (EPSG:4920). It's using GRS80 ellipsoid (not WGS84, but it's pretty close), and projected using a new datum that fits the entire Malaysia. I believe they are trying to push this new projection as the new standard in Malaysia. You can check it out here from this PDF: http://www.jupem.gov.my/jupemcms/wp-content/upload/circular/PKPUP3-2009.pdf
    4. There are standalone apps that converts stuff online. (I think I stumbled across an online converter for Malaysian RSO data before). Just need to find what works..

    Sorry for my long rant. Hope it helps :)

    ReplyDelete
    Replies
    1. Wahhhh. Thanks a million for this feedback. My next task will be to chew all that up but the greater challenge will be how to translate that for a non-technical person to understand which to me is more difficult than the earlier but if I can do it then I have learnt the fine Art of transferring knowledge.

      Delete
    2. Thank you very much for your very informative "rant!" :D It helped ME a lot --- I'm going to see how we can use our various open source GIS tools with these projections. Should be OK, as long as they are not RSO-type projections! --- Howard

      Delete
  7. No problem guys, glad to help. I'm new to GIS myself, so I came across the same problem too..

    Mr. Abbas: I agree, that's a really hard task - since there's a lot of technical jargon involved. Maybe need to do a lot more visual explaining than usual.. On the other note, I see that you teach QGIS classes: is there one that's open to public? Would love to learn QGIS (too lazy to learn by myself, hehe)

    Mr. Howard: Glad my rant helps, hehe. I strict myself to open-source tools and data too. Handling RSO looks like a lot of work, but handling Cassini must be more pain in the behind, right? Maybe that's why JUPEM is charging people for conversion services..

    ReplyDelete
    Replies
    1. Well I use anything I think of eventhough the choice sounds weird, for example, when I use GRASS modules in QGIS I go from alam nyata (QGIS) to alam ghaib (GRASS) then export the output back to alam nyata (QGIS). Particpants laugh but the key thing is that they understand. Yes, I teach QGIS and feel I understand how to go about it because participants are happy after the class though I never give out any documents because this forces them to concentrate and they do. My classes are very informal mainly because I don't like formality, in fact kaku. Those interested from public agencies need to write to my Ketua Pengarah for approval but when I get a green light, I proceed irrespective locally or some other state. Selain dari tu, kena lah hubungi secara peribadi.

      Delete
  8. Just like to chip in my experience with RSO in QGIS. There are 2 problems in the CRS for EPSG:3375 and EPSG:3168:

    1. The +gamma parameter is missing from the default crs so your map will be at an angle. QGIS gets its projection parameters from GDAL and this issue has already been fixed there - do a epsg_tr.py to generate the projection parameters. Somehow, QGIS srs.db is not picking this up. It still has the erroneous parameters. However, QGIS has corrected the projection to allow for the +gamma parameter since 1.7 I think.

    2. The projection used for RSO is omerc - Hotine Oblique Mercator. Malaysia uses Variant A of this projection (EPSG:9812) but QGIS applies the Variant B (EPSG:9815) from the PROJ 4 library which is finally being fixed I hear. So your projection will be a ways off. This issue has to do with the centre of projection applied.

    My workaround is to create a custom crs as follows:

    1. Add the +gamma parameter. I get the value from EPSG registry for Angle from Rectified to Skew Grid.
    2. Offset the False Easting x_0=472830.426 and False Northing y_0=442454.099. I calculated the values by reverse engineering the projection formula.

    The full custom crs is as follows:

    +proj=omerc +lat_0=4 +lonc=102.25 +alpha=323.013286728 +gamma=323.07483685 +k=0.99984 +x_0=472830.426 +y_0=442454.099 +ellps=GRS80 +units=m +no_defs

    Just remember to apply this custom CRS to your layers.

    ReplyDelete
    Replies
    1. Wah, you really throw me to the deep end of the swimming pool... without a float. Anyway, thank very much for the feedback. I invite anyone who has a better grasp of this subject than me to offer their views.

      Delete
  9. En. Hilmy is the one whose posts at the GDAL forums a year or two ago alerted me to the world-wide problem GDAL had with Hotine Oblique Mercator. This implies that perhaps our Michigan GeoRef may finally be modeled correctly by GDAL and by QGIS. Thank you for the info, En. Hilmy! --- Howard

    ReplyDelete
  10. Can any body answer me, if I have these coordinates it is in which coordinate system in MALAysia?
    I don't have the info about my data, around Kuala Lumpur X=417000,Y=350000
    Thank you

    ReplyDelete
    Replies
    1. The right authority to answer that is DBKL. I maybe wrong but I am made to be known that depts within DBKL do not practice the same CRS. If that is so, you've got a problem.

      Delete
  11. UTM-X UTM-Y
    280210 608010
    280640 613000
    276100 613300
    271790 612670
    268050 615130
    265540 617740
    272800 618500
    261610 617780
    278170 592750
    280220 588820
    281820 596950

    Hi, these are some coordinates related to a survey data by Jabatan Geology. They mentioned these are from Penang. But none them falling in the right place while plotting in the Google Earth software or other software. I used datum-WGS84, and Projection-NUTM47. Is there any probable solution? Please let me know. Thanks.

    ReplyDelete
    Replies
    1. Firstly, we must question if the coordinates given correct in the first place! Past experience has shown that this is not necessarily so even from respected people or agencies. So how to go about it?

      1.Go to www.boulter.com/gps
      2. Enter coordinates
      3. Select "Convert and map", 4 outputs will appear

      If the coordinates were correct, the actual location of the coordinates will show in the adjacent right map box.If not, sorry!

      This tool has never failed me.

      Delete